Tensor Product Multiscale Many-Particle Spaces with Finite-Order Weights for the Electronic Schrödinger Equation

We study tensor product multiscale many-particle spaces with finite-order weights and their application for the electronic Schrödinger equation. Any numerical solution of the electronic Schrödinger equation using conventional discretization schemes is impossible due to its high dimensionality. Therefore, typically Monte Carlo methods (VMC/DMC) or nonlinear model approximations like Hartree-Fock (HF), coupled cluster (CC) or density functional theory (DFT) are used. In this work we develop and implement in parallel a numerical method based on adaptive sparse grids and a particle-wise subspace splitting with respect to one-particle functions which stem from a nonlinear rank-1 approximation. Sparse grids allow to overcome the exponential complexity exhibited by conventional discretization procedures and deliver a convergent numerical approach with guaranteed convergence rates. In particular, the introduced weighted many-particle tensor product multiscale approximation spaces include the common configuration interaction (CI) spaces as a special case. To realize our new approach, we first introduce general many-particle Sobolev spaces, which particularly include the standard Sobolev spaces as well as Sobolev spaces of dominated mixed smoothness. For this novel variant of sparse grid spaces we show estimates for the approximation and complexity orders with respect to the smoothness and decay parameters. With known regularity properties of the electronic wave function it follows that, up to logarithmic terms, the convergence rate is independent of the number of electrons and almost the same as in the two-electron case. However, besides the rate, also the dependence of the complexity constants on the number of electrons plays an important role for a truly practical method. Based on a splitting of the one-particle space we construct a subspace splitting of the many-particle space, which particularly includes the known ANOVA decomposition, the HDMR decomposition and the CI decomposition as special cases. Additionally, we introduce weights for a restriction of this subspace splitting. In this way weights of finite order q lead to many-particle spaces in which the problem of an approximation of an N-particle function reduces to the problem of the approximation of q-particle functions. To obtain as small as possible constants with respect to the cost complexity, we introduce a heuristic adaptive scheme to build a sequence of finite-dimensional subspaces of a weighted tensor product multiscale many-particle approximation space. Furthermore, we construct a multiscale Gaussian frame and apply Gaussians and modulated Gaussians for the nonlinear rank-1 approximation. In this way, all matrix entries of the corresponding discrete eigenvalue problem can be computed in terms of analytic formulae for the one and two particle operator integrals. Finally, we apply our novel approach to small atomic and diatomic systems with up to 6 electrons (18 space dimensions). The numerical results demonstrate that our new method indeed allows for convergence with expected rates.Tensorprodukt-Multiskalen-Mehrteilchenräume mit Gewichten endlicher Ordnung für die elektronische Schrödingergleichung In der vorliegenden Arbeit beschäftigen wir uns mit gewichteten Tensorprodukt-Multiskalen-Mehrteilchen-Approximationsräumen und deren Anwendung zur numerischen Lösung der elektronischen Schrödinger-Gleichung. Aufgrund der hohen Problemdimension ist eine direkte numerische Lösung der elektronischen Schrödinger-Gleichung mit Standard-Diskretisierungsverfahren zur linearen Approximation unmöglich, weshalb üblicherweise Monte Carlo Methoden (VMC/DMC) oder nichtlineare Modellapproximationen wie Hartree-Fock (HF), Coupled Cluster (CC) oder Dichtefunktionaltheorie (DFT) verwendet werden. In dieser Arbeit wird eine numerische Methode auf Basis von adaptiven dünnen Gittern und einer teilchenweisen Unterraumzerlegung bezüglich Einteilchenfunktionen aus einer nichtlinearen Rang-1 Approximation entwickelt und für parallele Rechnersysteme implementiert. Dünne Gitter vermeiden die in der Dimension exponentielle Komplexität üblicher Diskretisierungsmethoden und führen zu einem konvergenten numerischen Ansatz mit garantierter Konvergenzrate. Zudem enthalten unsere zugrunde liegenden gewichteten Mehrteilchen Tensorprodukt-Multiskalen-Approximationsräume die bekannten Configuration Interaction (CI) Räume als Spezialfall. Zur Konstruktion unseres Verfahrens führen wir zunächst allgemeine Mehrteilchen-Sobolevräume ein, welche die Standard-Sobolevräume sowie Sobolevräume mit dominierender gemischter Glattheit beinhalten. Wir analysieren die Approximationseigenschaften und schätzen Konvergenzraten und Kostenkomplexitätsordnungen in Abhängigkeit der Glattheitsparameter und Abfalleigenschaften ab. Mit Hilfe bekannter Regularitätseigenschaften der elektronischen Wellenfunktion ergibt sich, dass die Konvergenzrate bis auf logarithmische Terme unabhängig von der Zahl der Elektronen und fast identisch mit der Konvergenzrate im Fall von zwei Elektronen ist. Neben der Rate spielt allerdings die Abhängigkeit der Konstanten in der Kostenkomplexität von der Teilchenzahl eine wichtige Rolle. Basierend auf Zerlegungen des Einteilchenraumes konstruieren wir eine Unterraumzerlegung des Mehrteilchenraumes, welche insbesondere die bekannte ANOVA-Zerlegung, die HDMR-Zerlegung sowie die CI-Zerlegung als Spezialfälle beinhaltet. Eine zusätzliche Gewichtung der entsprechenden Unterräume mit Gewichten von endlicher Ordnung q führt zu Mehrteilchenräumen, in denen sich das Approximationsproblem einer N-Teilchenfunktion zu Approximationsproblemen von q-Teilchenfunktionen reduziert. Mit dem Ziel, Konstanten möglichst kleiner Größe bezüglich der Kostenkomplexität zu erhalten, stellen wir ein heuristisches adaptives Verfahren zur Konstruktion einer Sequenz von endlich-dimensionalen Unterräumen eines gewichteten Mehrteilchen-Tensorprodukt-Multiskalen-Approximationsraumes vor. Außerdem konstruieren wir einen Frame aus Multiskalen-Gauss-Funktionen und verwenden Einteilchenfunktionen im Rahmen der Rang-1 Approximation in der Form von Gauss- und modulierten-Gauss-Funktionen. Somit können die zur Aufstellung der Matrizen des zugehörigen diskreten Eigenwertproblems benötigten Ein- und Zweiteilchenintegrale analytisch berechnet werden. Schließlich wenden wir unsere Methode auf kleine Atome und Moleküle mit bis zu sechs Elektronen (18 Raumdimensionen) an. Die numerischen Resultate zeigen, dass sich die aus der Theorie zu erwartenden Konvergenzraten auch praktisch ergeben

The octet rule in chemical space: Generating virtual molecules

We present a generator of virtual molecules that selects valid chemistry on the basis of the octet rule. Also, we introduce a mesomer group key that allows a fast detection of duplicates in the generated structures. Compared to existing approaches, our model is simpler and faster, generates new chemistry and avoids invalid chemistry. Its versatility is illustrated by the correct generation of molecules containing third-row elements and a surprisingly adept handling of complex boron chemistry. Without any empirical parameters, our model is designed to be valid also in unexplored regions of chemical space. One first unexpected finding is the high prevalence of dipolar structures among generated molecules.Comment: 24 pages, 10 figure

Parameterized Neural Networks for Finance

We discuss and analyze a neural network architecture, that enables learning a model class for a set of different data samples rather than just learning a single model for a specific data sample. In this sense, it may help to reduce the overfitting problem, since, after learning the model class over a larger data sample consisting of such different data sets, just a few parameters need to be adjusted for modeling a new, specific problem. After analyzing the method theoretically and by regression examples for different one-dimensional problems, we finally apply the approach to one of the standard problems asset managers and banks are facing: the calibration of spread curves. The presented results clearly show the potential that lies within this method. Furthermore, this application is of particular interest to financial practitioners, since nearly all asset managers and banks which are having solutions in place may need to adapt or even change their current methodologies when ESG ratings additionally affect the bond spreads.Comment: 24 pages, 17 figure

Predicting Properties of Oxide Glasses Using Informed Neural Networks

Many modern-day applications require the development of new materials with specific properties. In particular, the design of new glass compositions is of great industrial interest. Current machine learning methods for learning the composition-property relationship of glasses promise to save on expensive trial-and-error approaches. Even though quite large datasets on the composition of glasses and their properties already exist (i.e., with more than 350,000 samples), they cover only a very small fraction of the space of all possible glass compositions. This limits the applicability of purely data-driven models for property prediction purposes and necessitates the development of models with high extrapolation power. In this paper, we propose a neural network model which incorporates prior scientific and expert knowledge in its learning pipeline. This informed learning approach leads to an improved extrapolation power compared to blind (uninformed) neural network models. To demonstrate this, we train our models to predict three different material properties, that is, the glass transition temperature, the Young's modulus (at room temperature), and the shear modulus of binary oxide glasses which do not contain sodium. As representatives for conventional blind neural network approaches we use five different feed-forward neural networks of varying widths and depths. For each property, we set up model ensembles of multiple trained models and show that, on average, our proposed informed model performs better in extrapolating the three properties of previously unseen sodium borate glass samples than all five conventional blind models.Comment: 25 page

ATK-ForceField: A New Generation Molecular Dynamics Software Package

ATK-ForceField is a software package for atomistic simulations using classical interatomic potentials. It is implemented as a part of the Atomistix ToolKit (ATK), which is a Python programming environment that makes it easy to create and analyze both standard and highly customized simulations. This paper will focus on the atomic interaction potentials, molecular dynamics, and geometry optimization features of the software, however, many more advanced modeling features are available. The implementation details of these algorithms and their computational performance will be shown. We present three illustrative examples of the types of calculations that are possible with ATK-ForceField: modeling thermal transport properties in a silicon germanium crystal, vapor deposition of selenium molecules on a selenium surface, and a simulation of creep in a copper polycrystal.Comment: 28 pages, 9 figure

On the Interplay of Subset Selection and Informed Graph Neural Networks

Machine learning techniques paired with the availability of massive datasets dramatically enhance our ability to explore the chemical compound space by providing fast and accurate predictions of molecular properties. However, learning on large datasets is strongly limited by the availability of computational resources and can be infeasible in some scenarios. Moreover, the instances in the datasets may not yet be labelled and generating the labels can be costly, as in the case of quantum chemistry computations. Thus, there is a need to select small training subsets from large pools of unlabelled data points and to develop reliable ML methods that can effectively learn from small training sets. This work focuses on predicting the molecules atomization energy in the QM9 dataset. We investigate the advantages of employing domain knowledge-based data sampling methods for an efficient training set selection combined with informed ML techniques. In particular, we show how maximizing molecular diversity in the training set selection process increases the robustness of linear and nonlinear regression techniques such as kernel methods and graph neural networks. We also check the reliability of the predictions made by the graph neural network with a model-agnostic explainer based on the rate distortion explanation framework

Sparse grids for the Schrödinger equation

We present a sparse grid/hyperbolic cross discretization for many-particle problems. It involves the tensor product of a one-particle multilevel basis. Subsequent truncation of the associated series expansion then results in a sparse grid discretization. Here, depending on the norms involved, different variants of sparse grid techniques for many-particle spaces can be derived that, in the best case, result in complexities and error estimates which are independent of the number of particles. Furthermore we introduce an additional constraint which gives antisymmetric sparse grids which are suited to fermionic systems. We apply the antisymmetric sparse grid discretization to the electronic Schrödinger equation and compare costs, accuracy, convergence rates and scalability with respect to the number of electrons present in the system.